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SECTION I 
INTRODUCTION 

The Chief of Naval Technical Training (CNTECHTRA) Training Program 
Coordinator (TPC) for the Construction Electrician (CE) "A" School observed 
that remedial training in basic electricity and electronics (BE/E) informa- 
tion was desirable for students entering CE "A" School • One possible 
explanation for this need was that knowledge previously acquired at Navy 
BE/E Schools had not been well retained between the period following BE/E 
graduation and CE "A" School entry. Thus, the TPC (CNTECHTRA, Code N422), 
to assess this possibility, requested that the CE "A" Schools retest 
entering students. The Training Analysis and Evaluation Group (TAE6), with 
the concurrence of the Chief of Naval Education and Training (CNET), 
assisted CNTECHTRA by analyzing and interpreting the available data. 

PURPOSE 

The principal purpose of the CNTECHTRA/TAE6 effort was to assess the 
extent of knowledge decay during the interval between graduation from BE/E 
School and entry into a CE "A" School. Additional purposes of the study 
were to determine if knowledge decay 

• is related to the retention interval (i.e., time between schools) 

• is influenced by student ability characteristics 

• is affected by the BE/E School attended 

• is affected by. retest procedures at the CE "A" Schools entered 

• impacts on student CE "A" School achievement. 

In addition to examining the basic issue of BE/E knowledge decay, the 
TPC was also interested in determining if changes to the currently used 
selection criteria could improve student survivability within the CE 
training pipeline. Thus, an ancillary purpose of the study was to examine 
the probable results of alterations to current selection criteria. 

PERSPECTIVE ON RETENTION 

The issue of retention of learned technical skills and knowledges has 
long been of concern to the Navy. The extent to which acquired capabilities 
are forgotten over periods of nonuse has implications for many Navy programs 
and policies. In pipeline training programs, for example, knowledge losses 
over even relatively short periods of time may diminish the potential effec- 
tiveness of succeeding courses in the pipeline and/or require unprogrammed 
and unbudgeted remedial training. Personnel utilization policies and prac- 
tices must also contend with the retention problem. What happens to 
acquired capabilities when a seasoned individual is assigned a tour as a 
recruiter or when a new technical school graduate is assigned nonjob related 
duties for some period of time? Similar questions are of concern to those 
charged with structuring and manning the reserve forces. Are some critical 
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skills and knowledges more susceptible to decay than others? What steps 
should be taken to assure the availability of these capabilites in times of 
need? 

Current information about skill and knowledge decay is sketchy and 
incomplete. What is known stems largely from a small number of longitudinal 
studies which have provided limited empirical data about the retention of 
specific acquired capabilities. Although many retention phenomena could be 
predicted from the results of psychological learning experiments, the 
legitimacy of such generalizations to skills and knowledges essential for 
manning the Navy is unknown. In all, retention has been a relatively 
neglected area of investigation, and much information is needed to propose 
solutions to existing problems. The basic information needed concerns the 
retention problem itself; that is, the: 

• extent to which learned skills and knowledges may deteriorate or 
decay over periods of nonuse 

• types of skills and knowledges most/least susceptible to decay 

• rate(s) at which decay occurs. 

Information on these basic issues is prerequisite to the development of 
procedures, policies and practices for maintaining skills and knowledges at 
acceptable levels and to the specification of relearning requirements for 
forgotten capabilities. 

Definitive answers to the many questions about skills retention would 
require a long term programmatic effort. This type of effort, however, is 
prohibitive since it involves extensive costs for personnel testing and 
retesting and deliberate nonuse of personnel in trained-for specialties for 
varying periods of time. The present study is noteworthy for the 
opportunity it provides to develop an initial understanding of the problem 
of deterioration and to formulate realistic, practical approaches to 
contending with the problem over time. 

The present study provides important information that has been 
previously unavailable about the extent to which learned technical 
information is forgotten during periods of nonuse. The results obtained 
have direct implications for management of the CE training pipeline. They 
also have implications for other pipelines involving initial BE/E training. 
From a much broader perspective, this study is important to the Navy at 
large for the opportunity provided for an initial assessment of the nature 
of the problem of retention of acquired capabilities. 

ORGANIZATION OF THE REPORT 

The remainder of this report is presented in four sections and two 
appendices. Section II presents the technical approach of the study. The 
study results are given in section III and discussed in section IV. 
Conclusions and recommendations are provided in section V. A copy of the 
instructions provided the CE "A" School by CNTECHTRA for testing BE/E 
knowledge is contained in appendix A. Appendix B presents the results of a 
particular statistical analysis performed for the study. 
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SECTION II 



TECHNICAL APPROACH 

This section presents the technical approach to the study. The back- 
ground to the study is presented first. Subsequently, discussions are pro- 
vided of the basic study data, the variables examined and data analysis 
procedures employed. 

STUDY BACKGROUND 

Within the CE training pipeline, prospective CEs, immediately after 
completion of recruit training, attend a BE/E School at one of three Naval 
Training Centers: Orlando, FL; San Diego, CA; or Great Lakes, IL. 
Graduation from BE/E School requires successful completion of both phases 
a standardized comprehensive final examination. Phase 1 covers 
phase 2, AC theory. Following BE/E School, prospective CEs are 
a CE "A" School at either Port Hueneme, CA or Gulfport, 
that knowledge acquired at BE/E School may have decayed in 



(parts) of 
DC theory; 
ordered to 
Suspecting 



MS. 
the 

interval between BE/E graduation and entry into a CE "A" School, the 
responsible CNTECHTRA TPC (Code N422) requested that the CE "A" School 
staffs retest entering students. Subsequently, 307 BE/E graduates (covering 
the period between January and November 1981) were retested. Prior to the 
retest, students were given a 2-hour classroom period in which they could 
use BE/E study guides to refresh themselves on material covered in BE/E. An 
instructor was present to answer questions. No group lectures were 
provided. The following morning, students were given a multiple choice 
examination identical to the final examination taken at BE/E School. 
Students were encouraged to guess if they were not sure of the correct 
answer. The CNTECHTRA instructions for accomplishing the retesting are 
contained in appendix A. 

STUDY DATA 

All test data, and other data needed to address the study objectives, 
were provided TAEG by the CE "A" School TPC. The data included: 

identifying information on each student 

BE/E School attended 

location of the retest site (i.e., CE "A" School entered) 

final BE/E test, phase 1 scores (number correct out of a maximum 
score of 47) 

retest phase 1 scores 

final BE/E test, phase 2 scores (number correct out of a maximum 
score of 29) 

retest phase 2 scores 
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• scores of CE "A" School students on selected subtests of the Armed 
Services Vocational Aptitude Battery (ASVAB) 

• BE/E graduation date 

• BE/E retest date 

• CE "A" School final grades (these grades were available only for 
graduates of the CE "A" School in Gulfport). 

In addition to the above data, the TPC provided TAEG with ASVAB subtest 
scores on students selected to attend CE "A" School but who had attrited 
from BE/E School. This information was used for separate analyses concerned 
with possible improvement of criteria currently used to select individuals 
for CE "A" School training. 

STUDY VARIABLES 

The data provided by CNTECHTRA were used to compose variables for 
assessment. These variables, described below, are presented in the approxi- 
mate order of their occurrence in a CE "A" School student's history. The 
variables examined include student ability characteristics, the BE/E School 
attended, final BE/E test score grades, knowledge retention interval., the CE 
"A" School entered, BE/E retest score and CE "A" School grades. 

ABILITY CHARACTERISTICS. Relationships among measures of student ability, 
student achievement, and knowledge loss were examined. The ability measures 
used were AFQT percentile scores and ASVAB selector composite scores. 

AFQT Percentile Scores . Armed Forces Qualification Test (AFQT) percentile 
scores were used as measures of general ability. AFQT percentile scores, 
derived from three subtests of the ASVAB, are used in selecting recruits for 
entrance into the service. 

ASVAB Composite Scores . Composite scores made up from various combinations 
of ASVAB subtest scores are used by the Navy to select individuals for 
attendance at specific technical schools. Routine selection for CE "A 11 
training requires that a student's score on the Arithmetic Reasoning (AR) 
subtest plus twice his score on the Mechanical Knowledge (MK) subtest plus 
his score on the General Science (GS) subtest (i.e., AR + 2MK+ GS) equal or 
exceed 200. Analyses were conducted to (1) assess the relationships among 
composite scores and knowledge decay and (2) examine the effects of using 
different composites and different cut-off scores for CE "A" School 
selection. 

BE/E SCHOOL ATTENDED. As mentioned previously, BE/E training is conducted 
at Naval Training Centers in San Diego, Orlando, and Great Lakes. Location 
of the final BE/E test site (i.e., school attended) was used as a variable 
to determine its relationship to knowledge decay. Of specific concern was 
the equivalency of final test scores across the three sites. 
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FINAL BE/E TEST SCORE. Student total final test scores and scores on each 
phase of the BE/E final examination were used as separate criterion 
variables. The phase scores were used to determine if the predictor 
variables (e.g., ability levels, school attended) were differentially 
related to the specific knowledge measured by each test phase (i.e., DC 
theory for phase 1, AC theory for phase 2). 

RETENTION INTERVAL. The retention interval is the number of days between a 
student's final examination at BE/E School and his retest at CE "A" School. 
During this time, students could have been in transit or awaiting 
instruction. 

CE -A" SCHOOL ENTERED. After BE/E, students entered CE "A" School at either 
Port Hueneme or Gulfport. CE "A" School entered was used as a variable to 
determine if, for example, different retest practices at each school might 
affect an apparent knowledge decay. 

RETEST SCORE. The BE/E retest examination was identical to the final BE/E 
examination. The difference between the final BE/E test score and the 
retest score divided by the final test score reflected the proportion of 
knowledge lost over time for any given graduate. 

CE "A" SCHOOL GRADES. End-of -course grades were available only for those 
graduates who attended CE "A" School in Gulfport. These final grades, 
derived from objective and subjective measures, represent an average of test 
scores and a student's demonstrated performance in class. These school 
grades were used as a criterion variable to determine if any knowledge decay 
observed adversely affected achievement in CE "A" School. 

MISCELLANEOUS VARIABLES. In addition to the study variables discussed 
above, difference scores were also computed for each phase of the BE/E test 
and retest. The difference scores were computed by subtracting the final 
BE/E test scores from the retest scores. Therefore, a positive difference 
score indicates a BE/E graduate's retest score was higher than his final 
BE/E test score, while a negative difference score indicates a BE/E 
graduate's retest score was lower than his final BE/E test score. 

DATA ANALYSIS 

Analyses conducted were designed to address the following specific 
questions: 

1. Is there evidence of knowledge decay? If so, what is its 
relations', i p to: 

• retention interval 

• student ability characteristics 

• BE/E and CE "A" School sites 

• CE "A" School final grades? 
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2. What is a desirable ASVAB composite/cut -off score for selecting 
individuals for the CE pipeline? 

Regression analyses, analyses of variance and t-tests were used for 
standard data analyses. Discriminant function analysis was used to develop 
an alternative ASVAB composite. Trade-off curves were developed for both 
the currently used ASVAB composite and an alternative composite. 
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SECTION III 
RESULTS 

This section presents the results of the study. Presented first is a 
description of the total study sample. Second, descriptive information 
about BE/E graduates who went on to CE "A" School is provided. Next, the 
evidence for BE/E knowledge decay is given, and, finally, the relationships 
between CE "A" School grades, ability and knowledge decay are presented. 

STUDY SAMPLE 

A total of 381 BE/E students in the CE "A" pipeline were in the sample. 
Of these, 307 graduated from BE/E and went on to attend CE "A" School; 54 
attrited from BE/E for academic reasons and 20 attrited for nonacademic 
reasons. The entire sample, by BE/E School attended, is shown in table 1. 
Various analyses performed are based on smaller samples either because they 
concern only BE/E graduates or because of missing data. 



TABLE 1. SAMPLE OF BE/E STUDENTS 







Attrited 


CE "A" 


School 


BE/E School 


Academic 


Nonacademic 


Gulf port 


Port Hueneme 


Orlando 


21 


9 


99 


41 


San Diego 


23 


7 


14 


53 


Great Lakes 


10 


4 


50 


32 


Unidentified 


0 


0 


8 


10 


TOTALS 


54 


20 


171 


136 



DESCRIPTIVE INFORMATION ON CE "A" SCHOOL STUDENTS 

This subsection presents descriptive information on BE/E graduates who 
went on to attend CE "A" School. The mean AFQT percentile and ASVAB 
composite scores, BE/E test scores, and retention intervals are presented. 

AFQT AND ASVAB SCORES. The mean AFQT and ASVAB scores for BE/E graduates by 
BE/E and CE "A" School attended are shown in tables 2 and 3. There are no 
significant differences among any of the schools in AFQT scores or ASVAB 
composite scores. Thus, each BE/E School and each CE "A" School receives 
students of equivalent ability levels. 
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TABLE 2. AFQT PERCENTILE SCORES 



BE/E School 


CE 11 A" School 


Mean 


S.D.* 


Number 


Ureal LaKcb 




59.68 


13.30 


81 




Pnr»t" HiiPnPmP 
r uf i» nucMciiic 


57.56 


13.09 


32 




Gulfport 


61.06 


13.38 


49 


Can Hionn 




61.92 


18.18 


66 




DAwf UllAW AIWA 


61 39 


18.74 


52 




Gulfport 


63.93 


16.43 


14 


Orlando 




59.26 


17.08 


127 




Port Hueneme 


59.30 


16.60 


37 




Gulfport 


59.24 


17.36 


90 



*S.D. = standard deviation 

NOTE: Two-way Analysis of Variance showed no significant differences 
among BE/E Schools or CE "A" Schools. 
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TABLE 3. ASVAB COMPOSITE SCORES 



BE/E School CE "A" School 



Great Lakes 



Port Hueneme 
Gulfport 



Mean 



219.55 
218.23 
220.40 



S.D.* 



13.18 
12.31 
13.77 



Number 



82 
32 
50 



San Diego 



Port Hueneme 
Gulfport 



221.55 
221.32 
222.58 



16.48 
17.19 
13.47 



65 
53 
12 



Orlando 



Port Hueneme 
Gulfport 



219.14 
216.13 
220.70 



16.14 
14.43 
16.84 



117 
40 
77 



*S.D. = standard deviation 

NOTE: Two-way Analysis of Variance showed no significant differences. 
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BE/E TEST SCORES. Average final BE/E test scores are shown in table 4. 
There are no significant differences among final test scores by BE/E School, 
nor are there any differences between scores of students who went to either 
CE "A" School. Thus, the measured knowledge level of graduates of the three 
BE/E Schools is equivalent as is the knowledge level of graduates going to 
each CE M A" School site. 

Average BE/E retest scores are also shown in table 4. There are no 
significant differences in overall average retest scores between students at 
the two CE 11 A 11 Schools. This finding indicates that retest scores were not 
unduly influenced by retest practices at either school. However, there are 
significant differences in retest scores depending on which BE/E School was 
attended. Specifically, students who attended BE/E School in San Diego and 
CE "A" in Port Hueneme scored significantly higher on both phases of the 
retest than students from Orlando or Great Lakes who also attended CE 11 A 11 
School at Port Hueneme. However, there were no significant differences 
among students from the different BE/E Schools who attended CE "A" School at 
Gulfport. 

RETENTION INTERVAL. The average retention intervals in days for the 
different BE/E - CE "A" School pipelines are shown in table 5. When 
averaged over all students, there are no significant differences between the 
two CE "A" Schools in retention interval. However, there are significant 
differences in retention interval depending on the BE/E School attended. 
Specifically, students going from Orlando to Port Hueneme show the longest 
average interval (33.98 days) while students going from San Diego to Port 
Hueneme show the shortest average interval (18.23 days). 

KNOWLEDGE DECAY IN BE/E GRADUATES 

Average knowledge decay as the difference between the BE/E retest and 
final test is shown in table 4. Using paired t-tests, retest scores were 
found to be significantly lower than test scores (overall test, t - 16.01, 
£<.001; phase 1 test, t = 7.45, £<.001; phase 2 test, t = 19.6, £<.001). 

The proportion of knowledge decay, calculated as the difference between 
test and retest scores divided by the test score, was regressed on AFQT 
score and retention interval to determine the extent to which general 
intelligence and length of time between tests predict knowledge decay. The 
results of the regression analyses are shown in table 6. 

For both phases of the test, AFQT and interval significantly predict 
the proportion of decay in knowledge that occurs. AFQT has a negative 
coefficient with decay when the interval between tests is held constant, 
while interval has a positive coefficient when AFQT is held constant. Thus, 
as AFQT increased, decay would decrease, and as interval increased, decay 
would increase. 

RATE OF DECAY IN PHASE 1 AND PHASE 2. From table 4, it appears that more 
decay occurs in phase 2 (AC) than it does in phase 1 (DC). A paired t-test 
between the proportions of decay for phase 1 and phase 2 shows that there is 
a significantly greater proportion of decay in phase 2 knowledge (mean 
proportion = .24) than there is in phase 1 knowledge (mean proportion - .06, 
t = 16.9, p< .001). 
q 16 
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TABLE 4. AVERAGE BE/E TEST AND RETEST SCORES AND DIFFERENCE SCORES 



RE/E v-»hnn1 


f F "A" ^rhnnl 


FINAL BE/E TEST 
Raw Scores2 

Overall Phase 1 


SCORE 
Phase 2 


Overall 


BE/E RETEST 
Raw Scores 

Phase 1 


SCORE 

Phase 2 


RETEST-TEST DIFFERENCE 
Raw Scores 

Overall Phase 1 Phase 2 


Great Lakes 
(Nl 3 82) 


Port Hueneme 


60. 35 


39.55 


20.81 


50.34 


35.94 


14.41 


-10.01 


-3.62 


-6.4 




Gulf port 
(N = 50) 


61.22 


39.50 


21.72 


52.40 


37.62 


14.78 


-8.82 


-1.88 


-6.94 


San Diego 
(N = 65) 


Port Hueneme 
(N * 53) 


61.32 


40.06 


21.26 


58.75 


40.11 


18.64 


-2.59 


0.05 


-2.62 




Gulf port 
(N - 12) 


61.36 


39.64 


21.71 


55.50 


38.29 


17.21 


-5.85 


-1.35 


-4.5 


Orlando 
(N = 117) 


Port Hueneme 
(N = 40) 


6i. di 


40.05 


21.56 


49.34 


35.54 


13.80 


-12.27 


-4.51 


-7.76 




Gulf port 
(N = 77) 


62.23 


40.01 


22.22 


52.26 


37.24 


15.02 


-9.97 


-2.77 


-7.20 



*N is the number of graduates. 

2Maximum raw scores: overall - 76; phase 1 - 47; phase 2 - 29. 

Overall raw scores may not equal the summation of phase 1 and phase 2 raw scores due to rounding. 



IS 







Technical Report 


149 






TABLE 5. 


AVERAGE RETENTION INTERVAL IN DAYS 








CE "A 


" SCHOOL 






BE/E School 


Gulfport 


Port Hueneme 






Orlando 


28.96 
(97)* 


33.98 
(40) 






San Diego 


25.43 
(14) 


18.23 
(53) 






Great Lakes 


26.44 
(50) 


26.97 
(32) 






Average Interval 


27.87 


25.50 




* 


Number of students. 










TABLE 6. STANDARDIZED REGRESSION ANALYSES OF KNOWLEDGE 
ON AFQT AND RETENTION INTERVAL 


DECAY 




AFQT 


Interval 


R2 F 


Signi f . 


Phase 


1 -.172 


.259 


.10 14.8 


£<.001 


Phase 


2 -.144 


.379 


.17 27.1 


£<.001 



NOTE: All standardized coefficients are significant at the .01 level 
or less. Correlation between AFQT and Interval is -.027. 



Given the difference in proportion of decay between phase 1 and phase 
?, and the apparent difference between the regression coefficients of 
interval with phase 1 and phase 2 decay (table 6) , there might be a 
difference in the rate of decay in phase 1 and phase 2 scores. If this 
difference in rates exists, then, in a given amount of time, and for a fixed 
level of intelligence, the decay in phase 2 scores would be proportionally 
larger than the decay in phase 1 scores. This difference in decay rates 
would be indicated by a significant difference between the regression 
coefficients of retention interval with decay for phase 1 and phase 2. 

These two coefficients cannot be tested directly using a standard 
t-test since they are not independent. However, there is a test for 
dependent correlations (Cohen & Cohen, 1975). Since the multicolinearity 
between AFQT and Interval is so low (-.027), the beta coefficients should 
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act very similarly to correlation coefficients. This test shows that there 
is a significant difference between the correlation of Interval and phase 1 
decay and the correlation of Interval and phase 2 decay (t = 2.02, p<.05). 
Also, there is no difference between the analogous AFQT correlations (t = 
.354). These findings indicate that the rate of decay for phase 2 knowledge 
(AC theory) is greater than the rate of decay for phase 1 knowledge (DC 
theory) . 

PREDICTING CE "A" SCHOOL PERFORMANCE 

CE "A" School final grades were available for 88 graduates from 
Gulfport. A summary description of these grades is shown in table 7. 

To determine whether CE "A" School performance could be predicted by 
AFQT or ASVAB composite, and decay in knowledge, CE "A" School final grades 
were regressed on AFOT score and proportional decay in BE/E test scores and 
on ASVAB score and decay. Separate regressions were run for decay in total 
test score, phase 1 test score and phase 2 test score. The results are 
shown in tables 8 and 9. AFQT and ASVAB show a significant positive 
relationship with CE "A" final grade while the proportion of decay shows no 
significant relationship with CE "A" final grades. 





TABLE 7. GULFPORT 


CE "A" 


FINAL GRADES 






Range 


Min 


Max 


Mean 


Great Lakes 
(31)* 


23.44 


70.0 


93.44 


86.15 


San Diego 
(4) 


11.04 


79.87 


90.91 


87.60 


Orlando 

(53) 


26.1 


70.0 


96.1 


85.70 



*Number of students. 
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TABLE 8. STANDARDIZED REGRESSION OF CE "A" SCHOOL GRADE 
ON AFQT AND DECAY IN BE/E TEST SCORES 



Score Used 
for Decay 


AFOT 


Decay 


R2 


F 


Siqnif. 


Total Score 


.261* 


-.06 


.08 


3.5 


p <.03 


Phase 1 Score 


.261* 


-.08 


.08 


3.6 


p< .03 


Phase 2 Score 


.267* 


-.03 


.08 


3.4 


p< .04 



*These beta coefficients are significant at the .02 level. 



TABLE 9. STANDARDIZED REGRESSION OF CE "A" SCHOOL GRADE 

ON ASVAB COMPOSITE AND DECAY IN BE/E TEST SCORES 



Score Used 
for Decay 


ASVAB 


Dec ay 


R2 


F 


Signi f . 


Total Score 


.259* 


-.04 


.08 


3.5 


p< .03 


Phase 1 Score 


.256* 


-.07 


.08 


3.6 


p< .03 


Phase 2 Score 


.270* 


-.01 


.07 


3.4 


p<.04 



*These beta coefficients are significant at the .02 level. 
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SELECTION CRITERIA ANALYSES 

A separate statistical analysis was performed to determine how changes, 
to the currently used ASVAB criteria for selecting students for CE training 
would affect academic attrition rates in BE/E School. The results of this 
analysis are trade-off curves showing the percentages of erroneous decisions 
for different ASVAB composite cut-off scores. These curves are shown and 
discussed in the next section. 
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SECTION IV 
DISCUSSION 

This section is devoted to discussions of the findings related to BE/E 
knowledge decay, factors related to the decay, and the final "A" School 
grades of graduates. In addition, the probable effects of altering ASVAB 
selection criteria are discussed. 

KNOWLEDGE DECAY 

The finding that the retest scores were lower than scores on the final 
BE/E test indicates that there is a decay in BE/E knowledge. The average 
student lost about 24 percent of AC theory knowledge and about 6 percent of 
DC theory knowledge. 

Several explanations for these decay differences may be offered. 
Different types of knowledge (content) may have inherently different rates 
of decay. However, the observed differences in amount of decay could also 
be attributed to such factors as unequal learning of the two content areas, 
different test difficulties, or to situational aspects of the retest 
procedures at CE M A" School. The higher retest scores on phase 1 could have 
occurred because students learned the DC material better than they did the 
AC material. And, psychological learning theory asserts that higher degrees 
of learning lead to less forgetting (e.g., Hilgard and Bower, 1966). If 
unequal learning is a factor in AC knowledge being less well retained than 
DC knowledge, this in turn could be attributed to some combination of 
greater inherent difficulty of the AC area and instructional deficiencies. 

It is also possible that the observed differences in phase 1 and phase 
2 scores could be due to differences in the two test phases. The material 
covered in each phase could have been equally well learned, but a more 
difficult phase 2 (AC) test could have resulted in lower phase 2 scores both 
on the final test and on the retest. Finally, the study materials given the 
students prior to the retest could have favored DC knowledge to a greater 
extent than AC knowledge. Given more practice on DC items, student retest 
scores on that phase could have been raised relative to the AC part of the 
test. 

FACTORS RELATED TO KNOWLEDGE DECAY 

Several variables were investigated for their contributions to knowl- 
edge decay. These variables include: 

• the ability characteristics of a student 

• the retention interval (i.e., backlog time and/or transit time 
between the final test and retest) 

• the locations of the BE/E Schools and CE "A" Schools. 
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ABILITY CHARACTERISTICS. Higher ability students were found to have less 
decay of BE/E information than lower ability students with the same 
retention interval. Ihis finding suggests that knowledge decay could be 
lessened by raising ability level requirements to enter the traininq 
pipeline. This could be done by adjusting selection cut-off scores, and 
perhaps by changing the current ASVAB subtest selector composite. This 

a? P 1 C . ls A ll sc .!i? sed . be f !° w f . in 9 reater detail under the heading, "Predicting 
Academic Attrition in BE/E School." 9 

RETENTION INTERVAL. The number of days between the final BE/E test and the 
2!! V e L' r 1 e * ent1o J n interval) was the major factor influencing the 
amount of knowledge decay. This finding was substantiated when the 
influence of school location on the amount of knowledge decay was examined. 

School Location. There were no significant differences in overall averaqe 
retest scores between the CE "A" Schools; however, there were significant 
differences in the retest scores depending on which BE/E School was 
attended. Graduates who entered CE "A" School at Port Hueneme from San 

SecUl h ?L S T/F iC n nt J y , hl9h f aVerage reteSt scores leTs%n 0 °; ied S g a e n 

decay) than BE/E graduates from Orlando and Great Lakes. However the 

retest scores were not significantly different at the CE "A" School in 
Gulfport regardless of the BE/E School attended. 

One interpretation is that BE/E training at San Diego is superior to 
the training at the other BE/E Schools. However, this interpretation can be 
discounted since the results showed that the final BE/E test scores were 

ASvTl^ef 311 thre f BE/E l0Cati ° nS - In add i t i on , the mean AFQT and 
RF/F «hS2J. ^ S1 . mi . lar a cross 311 three BE/E Schools. Thus, all three 
BE/E Schools are training the same kind of student in terms of ability, and 
the output student quality level is uniform. Therefore, the alternative 
explanation is the difference in retention interval. BE/E graduates from 
San Diego who attended CE "A" School at Port Hueneme had the shortest 
average retention interval (i.e., 18 days), while BE/E graduates from 
[ S f a r n i G w\ t L , 3 , ke ? nad "^Jderably longer average retention intervals, 
ranging from 25 to 34 days depending on CE "A" School attended. 

Since retention interval is the most powerful predictor of knowledqe 
decay, a decrease in transit time between BE/E and CE "A" Schools should 

cr^ni n -n n ° Wled ?i deC3 /u. However ' 3 reduction of transit times involves 
SESS ling Problems that are created by unequal distances between the 
schoo s and by the irregularity of BE/E graduation dates relative to the 
starting dates of CE "A" Schools. The effect of unequal distances can best 
^Liw^ y C °-?, ar Ju 9 ^ knowle dge decay of the San Diego/Port Hueneme 
combination with the shortest travel distance and retention interval to 

ithPr 22hl2 d6Ca V f ° th o e c: r ,c BE/E Scn °°l-CE "A" School combinations. 'The 
other problem area (i.e. BE/E graduation and CE "A" starting dates) is a 
result of the different methods of instruction. BE/E students under 
computer-managed instruction (CMI) graduate individually, while CE "A" 

^nS?n^ R ?/^ n9 i r ? P ' l l aCed (GP) instruction, commence classes periodically. 
Sending BE/E graduates to the CE "A" School that convenes next would tend to 
minimize the retention interval and, therefore, the knowledge decay. 
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FINAL GRADES 

The final grades of Gulfport CE "A" graduates were used as measures of 
CE "A" School performance. Knowledge decay did not predict this 
performance. This finding, however, should not necessarily rule out the 
influence of knowledge decay on CE "A" performance. Whatever effects decay 
might have would likely show up in a student's early school performance. As 
time in CE "A M progresses, lost knowledge may be reacquired by the student 
through his own efforts or by the instructor's takinq time away from CE 
training to remediate the student. Thus, end-of -course grades would be 
insensitive to the decay effects. 

PREDICTING ACADEMIC ATTRITION IN BE/E SCHOOL 

The rdte of academic attrition in the current sample of BE/E students 
in the CE "A" School pipeline was 14 percent. During TAEG interactions with 
CNTECHTRA, some questions were raised concerning the appropriateness of both 
the current ASVAB composite makeup (i.e., subtests involved) and tne current 
cut-off score used for selection for the CE "A" pipeline. Discriminant 
function analysis (Nunnally, 1978) was used to determine if there was a more 
efficient set of composites and/or cut-off scores that could be used to 
discriminate between BE/E graduates and academic attrites. The results of 
the discriminant function analysis are shown in appendix B. 

Based on the discriminant function analysis, the three ASVAB subtests 
that discriminate best between BE/E graduates and academic attrites are 
mathematical knowledge (MK), electronics information (EI) and mechanical 
comprehension (MC). Based on the discriminant function coefficients, MK and 
MC were multiplied by a factor of two, so the best ASVAB composite based on 
the discriminant function analysis is EI + 2 (MK + MC). However, even 
though this new ASVAB composite discriminates between graduates and academic 
attrites slightly better than the old comDosite, the improvement is so 
slight as to be of no practical consequence. Therefore, instead of merely 
considering the ability of composite scores to predict graduation, the 
trade-off curves of various cut-off scores for both composites were ^so 
considered. 

SELECTION CRITERIA TRADE OFFS. In making a decision to admit a student into 
a training pipeline, there are two kinds of mistakes that can be made. A 
false positive mistake is admitting a student who later fails. A false 
negative mistake is not admitting a student who would have succeeded. In 
selecting a cut-off score there will almost always be a trade off between 
false positive and false negative decisons. A lower cut-off score will 
reduce the rate of false negative decisions, but it will increase the rate 
of false positive decisions. Conversely, a higher cut-off score will 
decrease the rate of false positive decisions but increase the rate of false 
negative decisions. Trade-off curves help to show the effects of changing 
cut-off scores on the rates of false positive and false negative decisions. 
However, trade-off curves do not necessarily show which particular cut-off 
point is the best one. A cut-off point should be selected based on the 
relative costs of false negative and false positive decisions. 
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Both false positive and false negative mistakes have costs attached to 
them. The cost of making a false positive decision is the time and money 
spent training a student who will not pass the course. This cost can be 
fairly easy to determine since it will be a function of the actual costs 
incurred by the student while in training. The cost of making a false 
negative decision is that a student who would have successfully finished 
training is not selected. Thus, a potential trained person is lost. The 
cost of this mistake is harder to determine; it will largely be a function 
of the availability of qualified personnel. 

Selecting a specific cut-off point for a composite ASVAB score (or any 
other entry criterion) should involve examination of a trade-off curve. 
Also, a decision must be made concerning a desirable mix of false positive 
and false negative decisions. In some cases, it will be desirable to reduce 
false positive decisions and in other cases it will be desirable to reduce 
false negative decisions. 

Reduction of false negative decisions, with a concommitant increase in 
false positive decisions, is desirable when the availability of qualified 
manpower is low relative to the number of personnel required. In this case, 
the cost of training people who will not complete the pipeline is offset by 
the need for a given quantity of trained personnel. This decision would 
necessitate lowering the ASVAB composite score required for entry into a 
training pipeline. 

Reduction of false positive decisions is desirable when the pool of 
available qualified manpower is large and the cost of training is high. 
Increasing the required composite entry score would increase false negative 
decisons and decrease false positive decisions. Thus, few students would 
enter the pipeline and subsequently fail, while more students who could have 
passed would be denied entry. This would result in lower training costs per 
students but is practical only if the qualified applicant pool is large. 

For the specific case of the CE "A 11 School pipeline, trade-off curves 
using both the current ASVAB composite and the new composite based on 
discriminant function analysis are shown in figures 1 and 2. 

With the ASVAB subtest composite and cut-off score currently used 
(i.e., 200), the total percent of wrong decisions is approximately 23 
percent. False positive decisions amount to about 10 percent and false 
negative decisions amount to about 13 percent. When the cut-off score is 
lowered to 190, as is the actual case when waived students are considered, 
the false positive rate increases to about 14 percent while the false 
negative rate falls to about 8 percent. Conversely, if the cut-off score 
were raised to 210, the rate of false positive decisions would fall to about 
7 percent while the rate of false negative decisions would increase to 33 
percent. 

The trade-off curve for the alternative composite shows essentially the 
same relationships between cut-off score and percent of false decisions but 
the rate of change in error rates is somewhat different. Specifically, 
raising the cut-off score does not result in as rapid an increase in false 
negative decisions for the alternative composite as it does for the current 
composite. 
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FALSE NEGATIVES 



FALSE POSITIVES 



25 



COMPOSITE SCORE (X 10) 

Figure 1. Trade-Off Curves for the Current ASVAB Composite 
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Fiqure 2. Trade-Off Curves for the Alternative ASVAB Composite 
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SECTION V 
CONCLUSIONS AND RECOMCNDATIONS 

CONCLUSIONS 

1. There is a decay in BE/E knowledge over time. Decay is greater 
for phase 2 knowledge (AC theory) than for phase 1 knowledge (DC theory). 
The average student lost about 24 percent of tested AC theory knowledge and 
about 6 percent of tested DC theory knowledge, 

2. The amount of knowledge decay increases as the time between tests 
increases. 

3. The amount of knowledge decay increases as student ability 
decreases. Over the retention intervals examined, higher ability students 
lost less knowledge than lower ability students. 

4. There were no differences in ability or BE/E test scores among the 
three BE/E Schools. Also, students at both CE "A M Schools had about the 
same overall average retest scores. However, students who attended BE/E 
School in San Diego and CE "A" School at Port Hueneme had higher retest 
scores and less knowledge decay. This was attributed to the much smaller 
time interval between tests for this group of students than for other 
groups. 

5. Students with higher ability tend to get higher final grades. 
Knowledge decay did not influence CE "A" final grade. However, final grades 
may not be sensitive to detecting decay effects. 

6. The current ASVAB composite is as good as any available composite 
for predicting success in BE/E School. Selecting a particular cut-off score 
is important. Trade-off curves are a useful means for doing this. 

RECOMMENDATIONS 

1. Determine the relative costs of false negative and false positive 
decisions for entry into the CE "A" pipeline. If there are large numbers of 
qualified students for entry into the pipeline, then consider raising the 
entry criteria. If there are fewer qualified students than openings, then 
consider lowering the criteria. 

2. Reduce the mean time delay between BE/E graduation and CE ''A 11 
School entry. This could be achieved by sending BE/E graduates to the CE 
"A" School with the earliest convening date. 

3. Even the most efficient pipeline will have delays between schools. 
Also, since the individualized BE/E School graduates students constantly and 
the CE "A" Schools class up periodically, some delay waiting for instruction 
is inevitable; therefore, BE/E students should be routinely tested for their 
retention of BE/E knowledge and given remediation if indicated. 
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4. The results of this study suggest that AC theory may be less well 
learned by BE/E students than DC theory. Further investigations are 
recommended to determine if the quality of AC instruction is equivalent to 
the quality of DC instruction. 

5. Continue, over the long term, investigations of knowledge loss and 
its relationship to subsequent learning and performance. A recommended 
initial effort in this area is to extend the present type of study to 
investigate knowledge loss for a large sample (2,000-3,000) of BE/E 
graduates going into other ratings. This investigation could confirm the 
results of the present study and permit better development of retention 
curves over longer periods of time. Part of this effort should also be 
devoted to determining absolute knowledge losses under conditions where 
students are not first provided opportunity to review BE/E material before 
being retested. 

6. Investigate knowledge losses for conventionally taught courses to 
determine if results similar to those for this CMI course are obtained. 



28 30 



Technical Report 149 



REFERENCES 

Cohen, J. and Cohen, P. Applied Multiple Regression/Correlation Analys 
for the Behavioral Sciences . Hillsdale, NJ: Lawrence Erlbaum 
Associ ates, 1975. 

Hilgard, E. R. and Bower, G. H. Theories of Learning . New York: 
Appleton-Century-Crofts, Inc., 1966. 

Nunnally, J. C. Psychometric Theory (2nd ed.), New York: McGraw-Hill, 
1978. 



31 

29 



Technical Report 149 



APPENDIX A 

INSTRUCTIONS FOR TESTING BASIC ELECTRICAL KNOWLEDGE 
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This appendix presents the instructions developed by CNTECHTRA for 
testing BE/E knowledge at the CE "A" Schools. 

INSTRUCTIONS FOR TESTING BASIC ELECTRICAL KNOWLEDGE 

Background 

Serious questions have been raised about the effectiveness and 
relevancy of the BE/E preparatory training given to prospective CE "A 11 
School students. This is a multifaceted problem requiring careful analysis 
if specific problem aspects are to be isolated, quantified, and remedied. 

Purpose 

As part of the larger analysis of the BE/E program, the thrust of this 
particular study (i.e., BE/E re-testing) will be to measure knowledge 
retention among BE/E graduates, evaluated against each of several variables 
which may affect knowledge retention. Since careful data collection in this 
regard can be significant in identifying methodology deficiencies in the 
present instructional system, it is vital that the testing and information 
processing procedures outlined herein be consistently applied.... 

Procedure 

1. When a student checks aboard NAVCONSTRACEN, a designated school 
official will extract the following information from that student's service 
record: name, social security number, BE/E school location, BE/E graduation 
date, and composite ASVAB Score (AR + 2 MK + GS). This information will be 
retained by this designated school official until receipt of the completed 
examination answer sheets from the CE "A" School instructor. (See Item 4 
below.) 

2. At the beginning of the sixth period of the second day following class- 
up, the instructor will make the following announcement to his CE "A" School 
students: "Tomorrow morning first period you will be given a multiple 
choice examination to test your knowledge of basic electricity. This test 
will cover essentially the same subject areas that you studied in BE/E 
School. This test is being given to ensure that you have an adequate 
knowledge of the principles of electricity to successfully complete 
Construction Electrician "A" School. A good knowledge of electricity is 
important to your success in the Navy because you will be using this 
knowledge every day, both as a student and as a CE." 

"While there will not be a 'Passing 1 or 'Failing 1 test score as such, 
the test results wi 1 1 be used to identify any of you who may need assistance 
in any of the basic electricity subject areas. You will have use of a Study 
Guide for the next two hours in this classroom to refresh yourself on the 
material you covered in BE/E. You will not be allowed to take this Study 
Guide away from the classroom this afternoon after class, so make good use 
of the time you have available. I will be available for the next two hours 
to answer any specific individual questions you may have, but I do not 
intend to conduct group lectures on any of the BE/E subject areas." Pass 
out Study Guides at this time. 
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Notes: 

a. This should be the first time that the students learn they will be 
tested on BE/E material. 

b. Students are not to be told that they will be taking the same 
identical examination that they took in BE/E. 

c. If demand warrants, two instructors should be available in the 
classroom during this two hour review period. 

d. Ensure that none of the students to be tested the following day is 
scheduled for some extraordinary activity (such as a mid-watch) on the 
evening of the second day, as this would bias the test results. 

3. On the morning of the third day (first period), the re-examination will 
be given. Have the students complete answer sheet heading items 1, 2, 3, 5, 
6, 8, 9, and 10 as described in Attachment (A) herewith (two answer sheets 
each) prior to passing out examination booklets. Tell the students "You 
will be allowed 1-1/2' hours to complete the test. If you don't know the 
answer to a particular test question, it's to your advantage to guess at it. 
You may now start the test." (Note to the Test Proctor: If a student does 
not comprehend a word or phrase within a test question, you may clarify the 
meaning of that word or phrase. You must be careful, however, to avoid 
"leading" the student to the selection of any particular answer choice.) 

4. After completing the examination and properly accounting for and 
securing all test booklets, the answer sheets are to be turned-in as 
designated by the school CISC The designated school official will then 
enter data elements 4 and 7 on each answer sheet, as described in Attachment 
(A), and will verify the accuracy of data elements 1, 5, 6, 9, and 10. The 
complete package of answer sheets for each CE "A" School class is to be 
mailed to CNTECHTRA Code N422 within five working days of the examination 
date. (Note: Prior to mailing the answer sheets to CNTECHTRA, they may be 
manually graded for local school use; however, care must be taken to avoid 
extraneous marks on these answer sheets, as they will be processed by ADP 
equipment at CNTECHTRA.) 
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ANSWER S 

Data Element 

1. Learning Center 

2 . Name 

3. Rate/Rank 

4. Test 

(See Instructions below) 

5. School 

6. Date 

7. Carrel! 

(See Instructions below) 

8. Social Security # 

9. Course # 

10. Test # 

Instructions 

With the exception of data el em 
all required test heading data prior 



ET HEADING 

Entry Instructions 

Enter "GP" or "PH," depending on 
CE "A" School location 

Self-explanatory 

Self-explanatory 

Enter student's ASVAB score as 
determined from Service Record 
(CE ASVAB = AR + 2 MK + GS) 

Enter "San Diego," Great Lakes," 
or "Orlando" depending on where 
student attended BE/E training 

Date of this particular test; 
e.g., "6 APR 81" 

Enter the date the student 
graduated from BE/E 

Self-explanatory 

Enter "71" 

Enter "021202" on first answer 
sheet (Exam Part I); enter 
"021203" on second answer sheet 
(Exam Part II) 



nts 4 and 7, the student should complete 
to commencing the examination 



ATTACHMENT (A) 
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APPENDIX B 



DISCRIMINANT FUNCTION ANALYSIS WITH ASVAB SUBTESTS AND 
GRADUATION OR ACADEMIC ATTRITION FROM BE/E 
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DISCRIMINANT FUNCTION ANALYSIS WITH ASVAB SUBTESTS AND 
GRADUATION OR ACADEMIC ATTRITION FROM BE/E 

Discriminant function analysis (Nunnally, 1978) is a technique for 
determining what function or functions of a set of dependent variables best 
discriminates between two or more groups of people or objects. in tms 
particular case, the independent variables are seven available ASVAB subtest 
scores and the groups are people who graduated from BE/E and people who 
attrited academically. Nonacademic attrites are ignored for the purpose of 
this analysis. 

Since there are three subtests in the currently used ASVAB composite, 
the best three subtests, as indicated by the magnitudes of the 
unstandardized discriminant function coefficients (table B-l), were selected 
for the alternative composite. These subtests are MK, EI, and MC. Also, 
since the coefficients for MK and MC are larger than the one for EI, these 
two are multiplied by a factor of 2. Thus, the alternative composite score 
is EI + 2 (MK + MC). 



TABLE B-l. DISCRIMINANT FUNCTION COEFFICIENTS FOR ASVAB SUBTEST 
AND BE/E ACADEMIC SUCCESS OR FAILURE 



ASVAB 
Subtests 


Discriminant 


Function Coefficients 


Standardized 


Unstandardized 


NO 


.225 


.032 


WK 


.165 


.026 


AR 


.156 


.027 


MK 


.312 


.060 


EI 


.323 


.044 


MC 


.545 


.076 


GS 


-.022 


-.003 


Note: Chi 2 = 


22.9, df = 7, p_<.002, n 


= 327. 



37 

35 



Technical Report 149 



DISTRIBUTION LIST 

Navy 

OASN (M&RA) 

CNO (OP-115, 0P-987H, OP-987, OP-11, OP-12) 
ONR (442 (3 copies), 270) 
CNM (MAT-0722) 
CNET (01, 02, N-5) 
CNAVRES (02) 

COMNAVAIRSYSCOM (03, 340F, 413E) 
CNTECHTRA (016 (5 copies), N-6) 
CNATRA (Library (2 copies)) 
COMTRALANT (00) 
COMTRALANT (2 copies) 
COMTRALANT (Educational Advisor) 
COMTRAPAC (2 copies) 
CO NAVPERSRANDCEN (Library (4 copies)) 
NAVPERSRANDCEN Liaison (021) 
Superintendent NAVP6SC0L (2124, 32) 

Superintendent Naval Academy Annapolis (Chairman, Behavioral Science Dept.) 

CO NAVEDTRAPRODEVCEN (Technical Library (2 copies), PDM) 

CO NAVEDTRASUPPCENLANT (N-3 (2 copies)) 

CO NAVEDTRASUPPCENPAC (2 copies) 

CO NAVAEROMEDRSCHLAB (Chief Aviation Psych. Div.) 

CO FLECOMBATRACENPAC 

CO FLECOMBATRACENLANT 

CO NAMTRAGRU 

CO NAVTECHTRACEN Corry Station (00, 101B, 3330, Cryptologic Training Department) 

CO NAVTRAEQUIPCEN (TIC, N-001, N-002, N-09P) 

Center for Naval Analyses (2 copies) 

OIC NODAC (2) 

CO TRITRAFAC (00, 02) 

CO NAVSUBTRACENPAC (2 copies) 

CO FLEASWTRACENPAC 

CO FLEASWTRACENLANT 

CO NAVSUBSCOL NLON (00, 0110) 

CO NAVTECHTRACEN Treasure Island (00, Technical Library) 

TAEG Liaison, CNET 022 (2 copies) 

DIR NAVEDTRAPRODEVCENDET Memphis 

CO NAVTECHTRACEN Meridian 

COMFLETRAGRU Pearl Harbor 

DIR NAVEDTRAPRODEVCENDET Meridian 

CNET Liaison Officer, Williams Air Force Base 

DIR NAVEDTRAPRODEVCENDET GLAKES 

CO, SERVSCOLCOM GLAKES (CISO, 00) 

CISO, NTTC Meridian 

CO NAVAEROSPMEDINST (Code 13, Code 11) 

CO FLETRACEN, Mayport 

38 

0 (Page 1 of 2) 

ERIC 



Technical Report 149 
DISTRIBUTION LIST (continued) 

Air Force 

Headquarters, Air Training Command (XPTD, XPT1A) Randolph Air Force Base 
Air Force Human Resources Laboratory, Brooks Air Force Base (2 copies) 
Air Force- Human Resources Laboratory (Library), Lowry Air Force Base 
Air Force Office of Scientific Research/NL 

Headquarters Tactical Air Command (DOOS), Langley Air Force Base 

AFMTC/XR, Lackland Air Force Base 

Headquarters 34 TATG/IDM, Little Rock Air Force Base 

Headquarters MAC/DOT, Scott Air Force Base 

4235 Strategic Training Squadron, Carswell Air Force Base 



Army 

Commandant, TRADOC (Technical Library) 

ARI (Technical Director, PERI-RH, PERI-SM, PERI-IC, Library (2 copies)) 

ARI Field Unit - Fort Leavenworth 

ARI (Reference Service) 

ARI Field Unit - Fort Knox (PERI-IK) 

COM USA Armament Materiel Readiness Command (DRSAR-MAS) 

COMDT, USAIPRM (ATSG-DT-R) 

USA Missile & Munitions Center and School (ATSK-AIT) 



Coast Guard 

Commandant, Coast Guard Headquarters (G-P-l/2/42, GRT/54) 



Marine Corps 

CMC (OT) 
CGMCDEC 

Director, Marine Corps Institute 
CO MARCORCOMMELECSCOL 



Other 

Military Assistant for Human Resources, OUSDR&E, Pentagon 

Institute for Defense Analyses 

COM National Cryptologic School (Code E-2) 

Old Dominion University 



Information Exchanges 

DTIC (12 copies) 
DLSIE 

Executive Editor, Psychological Abstracts, American Psychological Association 
ERIC Processing and Reference Facility, Bethesda, MD (2 copies) 



ERLC 



(Page 2 of 2) ^9 



